? freescale semiconductor, inc., 2004. all rights reserved. this document contains informatin on a produ ct under development. freescale reserves the right to change or discontinue this product without notice. freescale semiconductor white paper IMXHANTROWP/d rev. 2, 09/2004 executive briefing although multimedia is on the tip of everybody's tongue when we talk about the advanc ed capabilities of new wireless handheld communications devi ces, the technologies that can really turn multimedia serv ices into useful tools and eye-popping entertainment are just beginning to reach the marketplace. the i.mx1 is the latest of the advanced class of arm9? core-based applications proc essors that are designed to specifically address requiremen ts of turning full-blown, real-time multimedia into a poc ket-sized reality. the i.mx1, with an arm920t? core processo r, is designed to provide performance with stamina ? a powerful, cost -effective and low-power applications processor for smart phones, wireless enabled pda's, and other mobile wireless devices. i.mx1's integrated multimedia functions include a color lcd controller, which is designed to support up to vga (640 pixels x480 lines) resolution, and video port with built-in image pre-processing. in ad dition to the arm920t core, freescale semiconductor has also added a fixed-point multiplier and dct (discrete cosine transform) hardware accelerator. the combination of the arm920t core processor and freescale technology enables the i.mx1 to readily meet the performance requirements needed to implement today's multim edia applications. i.mx applications processors with hantro's multimedia framework by: clint powell , freescale semiconductor inc. marko nurro , hantro products oy
i.mx application processors with hantro?s multimedia framework white paper, rev. 2 2 freescale semiconductor freescale, however, has gone a step fu rther to extend the multimed ia capabilities of the i.mx family of processors. the company has teamed with hantro and licensed th eir advanced video codec. the hantro codec may be implemented as software, hardware, or as a combination of software and hardware. the hantro software codec is included in the i.mx1 delivery package. i.mx family and hantro video technology having already ported the hantro soft ware video codec to the i.mx1 processo r, we have now added hantro video codec hardware, along with the soft ware, to our recently announced ne xt generation i.mx 21 applications processor. designed to deliver real -time mpeg4 and h.263 video compressio n and decompression, the codecs are standards-based and scalable from low image re solutions and frame rates up to ci f (352x288) at 30 fps (frames per second). further more, to facilitate applicatio n development, multimedia middleware is available from hantro to enable functions such as: video record, pla yback (in a camcorder, for instance) a nd content sharing through standard compression mechanisms and file form ats. the high compression ratios of the video codec helps ensure compact video file sizes, designed to deliver a cost-effective us e for the end user when sharing content over a wireless network. i.mx21
i.mx application processors with hantro?s multimedia framework white paper, rev. 2 freescale semiconductor 3 the arm920t processor in the i.mx1 has plenty of perfo rmance overhead to perform sophisticated multimedia functions using hantro's software codec. for instance, running video conference?with viewfinder, mpeg-4 decode, image deblock, image csc (color space conv ersion), and mpeg-4 encode-on qcif (quarter cif-176x144), 15 fps at 64kbits/sec?uses only about 150 mh z (speeds are approximate) with a 16-bit external memory bus. a 32-bit external memory bus helps reduce the speed and performan ce requirements even further, to about 130 mhz. in another example, one-way video streaming of qcif at 30 fps, using mpeg-4 decode , image deblock, and image csc, requires just over 100 mhz with a 16-bit memory bus and only abou t 90 mhz with a 32-bit memory bus. both the hantro codec and the i.mx1 ar chitecture are optimized to provide hi gh performance at low power levels. so, even though software solutions are often viewed as re source hogs, in this particular case the flexibility of a software codec doesn't stretch the processor to the limits of its performance capabilities. i.mx21 and the hantro hardware implementation the next generation i.mx processor, the i.mx21, includes the arm926ej-s? core processor operating at speeds starting at 266 mhz, and is designed to support both 16-b it and 32-bit external memory buses. the i.mx21 also includes an lcd controller supporti ng resolutions up to svga resolu tions, emma (enh anced multimedia accelerator), and a video port. the i.mx21's emma includes the hant ro mpeg-4 hardware and freescale technology for pre-processing and post-processing hardware. u tilizing the i.mx21 emma for a videophone use case scenari o (see below), less than 20 mhz of processor loading may be requ ired to support these functions ther eby helping to reduce the overall use case loading which is described later. there are distinct advantages offere d by the hardware emma. all comput er intensive encoding and decoding calculations are executed in hardware, with software only hand ling the control functions . this means that the i.mx21 is designed to handle the simultaneous encode and decode of cif resolution video at 30 fps. this kind of performance is particularly critical to meet the dema nding requirements of real-time 30 fps wireless video conferencing. the i.mx21 is designed to meet these requ irements and does it with resources to spare?resources that may be used to pe rform additional activities. also, by complementing hantro's vide o codec with freescale's pre-proces sing and post-processing hardware technology, the i.mx21 is a unique solution with enhanced multimedia pe rformance. the pre-processing hardware technology comprises image resizing and csc between the image sensor and the encoding process. the post-processing hardware technology comprises ever ything between decoding and display. the freescale technology includes algorithms for deblocking, deringing, and resizing, which contribute to creating a sharper, clearer image, regardless of format
i.mx application processors with hantro?s multimedia framework white paper, rev. 2 4 freescale semiconductor in addition, there is an inherent advantage to implem enting the hantro mpeg-4 an d h.263 codec with freescale technology in hardware since it requires far less power to op erate than a software only so lution. simply put, it is designed to take less power to trip a ga te than it does to run software. still, the options are not simply all software or all hardware. the hantro video codec may be impl emented as a combination of hardware and software, depending upon applications processor priorities. two freescale test case analyses, one with a video phone and the other with one-way streaming, demonstrate the flexibility of the i.mx21/hantro solution, and how a ha rdware implementation can he lp reduce the amount of processing power needed for multimedia functions. a power saving demonstration in the videophone use case scenario (q cif, 15 fps), a live, real-time video and audio link is established between a wireless handheld and another device. the applications processor had to support th e following applications, running in parallel: mpeg-4 encode ; mpeg-4 decode; mpeg-4 post-processing, including deblock, dering, resizing, and color space conversion; image pre-processing and camera view finding; amr (adaptive multi-rate) voice encode; amr voice decode; echo cancellation, and op erating system overhead, us ing operating systems such as symbian os, palm os, wince or elinux. all megahert z numbers are approximate and are based on simulated performance analyses. to run all these functions using only the arm926ej-s core unit would require well in excess of 400 mhz. in this case, mpeg-4 encoding alone would require about 190 mhz. employing lo w-level assembly optimized mpeg-4 encode and decode, plus emma pre-processing and post-p rocessing hardware acceleration, helps to reduce cycle usage to approximately 250 mhz. by implementing encode and decode in hardware, as well as the fre escale post-processing technology, performance requirements plummet to about 160 mhz. the one-way streaming use case scenario involves st reaming cif resolution 30 fps mpeg-4 encoded data from external memory/flash memory (or across a wireless link through an mp eg-4 decoder) to a display. the applications processor is required to support: mpeg-4 decode; mpeg post-processing, including deblock, dering, scaling, and color space conversion; audio decoder (mp3 or amr); and os overhead. for the arm926ej-s alone, using optimized mpeg-4 deco de software, meeting these requirements would require about 610 mhz. mpeg-4 decoding alone would take 140 mhz (much more than for the video phone because of the higher resolution and greater fra me rate requirem ents). by implementing the freescale post-processing technology in hardware, the performance requirement drops to about 210 mhz. furthermore, employing hantro's hardware decoder rather than the optimized decode soft ware-only solution gets you down to less than 100 mhz. the modular solution each block that makes up the emma?video encoder, video decoder, post-proces sing and pre-processing technology?is an independent componen t that may be implemented in hardwa re or software, re gardless of how the other blocks are employed.
i.mx application processors with hantro?s multimedia framework white paper, rev. 2 freescale semiconductor 5 for instance, the i.mx21 is designed to employ th e full hantro codec and the freescale pre-processing and post-processing in hardware, but the processor has plen ty of megahertz overhead to explore other options, depending on the application requirem ents. with a combinatio n software and hardware implementation of the hantro video codec, computation intensive priorities may be performed in hardware while less demanding and frequently changing requirements may be executed in soft ware, thus helping to prov ide power saving performance and enhanced flexibility exac tly where they are needed. the post-processing hardware acceleration is designed to su pport either the hardware decoder or a custom software decoder. even the sub-blocks within the post-processi ng block?deblock, dering, and resize?are independent components that may be individually bypassed by software configuration if it's not needed for a particular post-processing process use case scenario. the modular approach employed by both hantro and freescale also help s ease the migration from i.mx1 to i.mx21. since it's made up of independent components, th e software codec used in the i.mx1 may be transferred to the i.mx21 and plugged in to its new host. what's more, hantro provid es the multimedia middleware that ties all the blocks together for developing multimedia applications. in the end, re-development time and expenses are minimized when migrating from i.mx1 to i.mx21 allowing developers to reach the market quickly with new products. conclusion the combination of freescale?s i.mx processors?s high performance technology an d hantro?s software or hardware solution is ideal for making the most from available resources. the hantro software codec can provide a flexible implementation that enables the i.mx1 to meet th e needs of today's multimedia enabled wireless and hand held devices. now, stepping into the future with the powerful i.mx21, a full hardware implementation is possible, or a software/hardware combination can be used, employing the gr eatest advantages of each where they are needed the most. the combination of high performan ce, enhanced flexibility, and significan t power savings can be realized in one applications processor.
IMXHANTROWP/d rev. 2 09/2004 how to reach us: usa/europe/locations not listed: freescale semiconductor literature distribution center p.o. box 5405 denver, colorado 80217 1-800-521-6274 or 480-768-2130 japan: freescale semiconductor japan ltd. technical information center 3-20-1, minami-azabu, minato-ku tokyo 106-8573, japan 81-3-3440-3569 asia/pacific: freescale semiconductor hong kong ltd. 2 dai king street tai po industrial estate tai po, n.t., hong kong 852-26668334 home page: www.freescale.com information in this document is provided solely to enable system and software implementers to use freescale semiconductor products. there are no express or implied copyright licenses granted hereunder to design or fabricate any integrated circui ts or integrated circuits based on the information in this document. freescale semiconductor reserves the right to make changes without further notice to any products herein. freescale semiconductor makes no warranty, representation or guarantee regarding the suitability of its products for any particular purpos e, nor does freescale semiconductor assume any liability arising out of the application or use of any product or ci rcuit, and specifically disclaims any and all liability, including without limitation consequential or incidental damages. ?typical? parameters that may be provided in freescale semiconductor dat a sheets and/or specifications can and do vary in different applications and actual performance may vary over time. all operating parameters, including ?typicals?, must be validated for each customer application by customer?s technical experts. freescale semiconductor does not convey any license under its patent rights nor the rights of others. freescale semiconductor products are not designed, intended, or authorized for use as components in systems intended for surgical implant into the body, or other applications intended to support or sustain life, or for any other application in which the failure of the freescale semiconductor product could create a situation where personal injury or death may occur. should buyer purchase or use freescale semiconductor products for any such unintended or unauthorized application, buyer shall indemnify and hold freescale semiconductor and its officers, employees, subsidiaries, affiliates, and distributors harmless against all claims, costs, damages, and expenses, and reasonable attorney fees arising out of, directly or indirectly, any claim of personal injury or death associated with such unintended or unauthorized use, even if such claim alleges that freescale semiconductor was negligent regarding the design or manufacture of the part. learn more : for more information about freescale products, please visit www.freescale.com. freescale? and the freescale logo are trademarks of freescale semiconductor, inc. arm and the arm powered logo are the registered trademar ks of arm limited. arm920t and arm926ej-s are the trademarks of arm limited. freescale semiconductor inc. is an equal opportunity/affirmative action employer. ? freescale semiconductor, inc. 2004. all rights reserved. delivering real-time multimedia in pock et-sized mobile devices is a huge cha llenge, particularly since the world's carriers keep enticing the consumers with new and advanc ed video and audio services. the freescale family of i.mx processors is continuing to evolve to support the multimedia features of th e future. as part of this evolution, hantro's video codec provides a founda tion that enables rapid concept-to-mar ket implementation through the next several generations of the i.mx family of processors. revision history this revision is to provide change from motorola to freescale semiconductor.
|
